Lens array camera and method of driving lens array camera

ABSTRACT

A method of driving a lens array camera may include simultaneously driving a first group of sensing elements from among a plurality of sensing elements, each sensing element from among the first group of sensing elements corresponding to a same original signal viewpoint, wherein the plurality of sensing elements is included in a sensor corresponding to the lens array camera including N rows of N lenses, N being a natural number.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2018-0159958, filed on Dec. 12, 2018in the Korean Intellectual Property Office, the disclosure of which isincorporated by reference herein in its entirety.

BACKGROUND 1. Field

Methods and apparatuses consistent with example embodiments relate to alens array camera and a method of driving the lens array camera.

2. Description of the Related Art

Due to development of optical technologies and image processingtechnologies, image capturing apparatuses are being utilized in a widerange of fields, for example, multimedia content, security andrecognition. A size of an image capturing apparatus may be determinedbased on, for example, a size of a lens, a focal length of a lens or asize of a sensor. For example, the size of the image capturing apparatusmay be adjusted based on a size of a lens or a size of a sensor. As thesize of the sensor decreases, an amount of light incident on the sensormay decrease. Accordingly, a resolution of an image may decrease, or itmay be difficult to perform capturing in a low illuminance environment.To reduce the size of the capturing apparatus, a multi-lens includingsmall lenses may be used.

The capturing apparatus using the multi-lens may be mounted on a mobiledevice, a camera, a vehicle, and a computer due to its small volume andmay be used for acquiring an image, recognizing an object, orcontrolling a device.

SUMMARY

One or more example embodiments may address at least the above problemsand/or disadvantages and other disadvantages not described above. Also,the example embodiments are not required to overcome the disadvantagesdescribed above, and an example embodiment may not overcome any of theproblems described above.

In accordance with an aspect of the disclosure, a method of driving alens array camera, the method includes simultaneously driving a firstgroup of sensing elements from among a plurality of sensing elements,each sensing element from among the first group of sensing elementscorresponding to a same original signal viewpoint, wherein the pluralityof sensing elements is included in a sensor corresponding to the lensarray camera including N rows of N lenses, N being a natural number.

A number of the plurality of sensing elements and a number of the lensesmay be relatively prime.

The number of the plurality of sensing elements may be one more than apredetermined natural multiple of N, N being the number of rows oflenses.

The plurality of sensing elements may be arranged in (N*M+1) rows, Mbeing a predetermined natural number, wherein each row of the pluralityof sensing elements includes (N*M+1) sensing elements, wherein when p isequal to N/2 rounded up to the nearest natural number, the first groupof sensing elements includes a plurality of rows of sensing elementsgiven by

$\quad\{ \begin{matrix}{{{i \times M} + r},} & {0 \leq i < p} \\{{{i \times M} + r + 1},} & {p \leq i < N}\end{matrix} $

wherein r is a natural number corresponding to the first group ofsensing elements.

The first group of sensing elements may be one from among a plurality ofgroups of sensing elements, and the natural number r may be incrementedby 1 for each group of sensing elements from among the plurality ofgroups of sensing elements to be simultaneously driven.

Each lens from among the N rows of N lenses may cover a fraction of atleast one sensing element from among the plurality of sensing elementsthat is less than an entirety of the at least one sensing element.

The fraction of the at least one covered sensing element may be aninteger multiple of 1/N.

The method may further include outputting an image corresponding tooriginal signal information received by the plurality of sensingelements by restoring sensing information obtained using thesimultaneously driven first group of sensing elements.

In accordance with an aspect of the disclosure, a non-transitorycomputer-readable storage medium stores instructions that, when executedby a processor, cause the processor to perform a method in accordancewith the above-noted aspect of the disclosure.

In accordance with an aspect of the disclosure, a lens array cameraincludes a processor; and a memory including instructions to be read bya computer, wherein when the instructions are executed in the processor,the processor is configured to simultaneously drive a first group ofsensing elements from among a plurality of sensing elements, eachsensing element from among the first group of sensing elements beingpositioned at a same position relative to a respective lens, theplurality of sensing elements corresponding to the lens array cameraincluding N rows of N lenses, N being a natural number.

A number of the plurality of sensing elements and a number of the lensesmay be relatively prime.

The number of the plurality of sensing elements may be one more than apredetermined natural multiple of N, N being the number of rows oflenses.

The plurality of sensing elements may be arranged in (N*M+1) rows, Mbeing a predetermined natural number, wherein each row of the pluralityof sensing elements includes (N*M+1) sensing elements, wherein when p isequal to N/2 rounded up to the nearest natural number, the first groupof sensing elements comprises a plurality of rows of sensing elementsgiven by

$\quad\{ \begin{matrix}{{{i \times M} + r},} & {0 \leq i < p} \\{{{i \times M} + r + 1},} & {p \leq i < N}\end{matrix} $

wherein r is a natural number corresponding to the first group ofsensing elements.

The first group of sensing elements may be one from among a plurality ofgroups of sensing elements, and the natural number r may be incrementedby 1 for each group of sensing elements from among the plurality ofgroups of sensing elements to be simultaneously driven.

Each lens from among the N rows of N lenses may cover a fraction of atleast one sensing element from among the plurality of sensing elementsthat is less than an entirety of the at least one sensing element.

The fraction of the at least one covered sensing element may be aninteger multiple of 1/N.

The processor may be further configured to output an image correspondingto original signal information received by the plurality of sensingelements by restoring sensing information obtained using thesimultaneously driven first group of sensing elements.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a block diagram illustrating a lens array camera according toan example embodiment;

FIG. 2 is a diagram illustrating a relationship between a number ofsensing elements and a number of lenses according to an exampleembodiment;

FIG. 3 is a diagram illustrating a relationship between a number ofsensing elements and a number of lenses according to an exampleembodiment;

FIG. 4 is a diagram illustrating an arrangement state of a lens and asensing element according to an example embodiment;

FIG. 5 is a flowchart illustrating a lens array camera driving methodperformed by a lens array camera according to an example embodiment; and

FIGS. 6 and 7 are diagrams illustrating a device in which a lens arraycamera is implemented according to an example embodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to example embodiments, examples ofwhich are illustrated in the accompanying drawings, wherein likereference numerals refer to the like elements throughout. Exampleembodiments are described below in order to explain the presentdisclosure by referring to the figures.

The following structural or functional descriptions merely describe theexample embodiments, and the scope of the example embodiments is notlimited to the descriptions provided in the present specification.Various changes and modifications can be made thereto by those ofordinary skill in the art.

Although terms of “first” or “second” are used to explain variouscomponents, the components are not limited by the terms. These termsshould be used only to distinguish one component from another component.For example, a “first” component may be referred to as a “second”component, or similarly, and the “second” component may be referred toas the “first” component.

It will be understood that when a component is referred to as being“connected to” another component, the component can be directlyconnected or coupled to the other component or intervening componentsmay be present.

As used herein, the singular forms are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It shouldbe further understood that the terms “comprises” and/or “comprising,”when used in this specification, specify the presence of statedfeatures, integers, steps, operations, elements, components or acombination thereof, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

Unless otherwise defined herein, all terms used herein includingtechnical or scientific terms have the same meanings as those generallyunderstood by one of ordinary skill in the art. Terms defined in generaldictionaries should be construed to have meanings matching contextualmeanings in the related art and are not to be construed as an ideal orexcessively formal meaning unless otherwise defined herein.

FIG. 1 is a block diagram illustrating a lens array camera according toan example embodiment.

A quality of an image acquired and processed by a lens array camera 100may be determined based on a number of sensing elements included in asensor 120 and an amount of light incident on a sensing element. Forexample, a resolution of the image may be determined based on the numberof sensing elements included in the sensor 120 and a sensitivity of theimage may be determined based on the amount of light incident on thesensing element. The amount of light incident on the sensing element maybe determined based on a size of the sensing element. As the size of thesensing element increases, the amount of light incident on the sensingelement may increase, and a dynamic range of the sensor 120 mayincrease. Thus, as the number of sensing elements included in the sensor120 increases, the sensor 120 may acquire a higher resolution image.Also, as the size of the sensing element increases, the sensor 120 mayimprove the quality of high sensitivity imaging in a low lightcondition.

A size of the lens array camera 100 may be determined based on a focallength f of a lens 110. For example, the size of the lens array camera100 may be determined based on a gap between the lens 110 and the sensor120. To collect light refracted by the lens 110, the sensor 120 may belocated within the focal length f of the lens 110. Thus, the lens 110and the sensor 120 included in the lens array camera 100 may be spacedapart by a distance within the focal length f of the lens 110. The focallength f of the lens 110 may be determined based on a viewing angle ofthe lens array camera 100 and a size of the lens 110, for example, aradius of an aperture of the lens 110. When the viewing angle is fixed,the focal length f may increase proportionally to the size of the lens110. Also, the size of the lens 110 may be determined based on a size ofthe sensor 120. For example, to acquire an image in a predeterminedviewing angle range, the size of the lens 110 may increase as the sizeof the sensor 120 increases.

As described above, the size of the lens array camera 100 may beincreased to increase the sensitivity of the image while the viewingangle and the resolution of the image are maintained. For example, toincrease the sensitivity of the image while maintaining the viewingangle and the resolution of the image, the number of sensing elementsincluded in the sensor 120 may be maintained and the size of each of thesensing elements may be increased, which may increase the size of thesensor 120. In this example, in order to maintain the viewing angle, thesize of the lens 110 may be increased according to an increase of thesize of the sensor 120 and the focal length f of the lens 110 may beincreased, which may increase the size of the lens array camera 100.

To reduce the size of the lens array camera 100, a method of reducingthe size of the sensing element and maintaining the resolution of thesensor 120 or a method of reducing the resolution of the sensor 120 andmaintaining the size of the sensing element may be used. When reducingthe size of the sensing element and maintaining the resolution of thesensor 120, the size of the sensor 120 and the focal length f of thelens 110 may be reduced. In this case, the size of the lens array camera100 may be reduced but the sensitivity of the image may also be reduced.Thus, a low-illuminance image quality may be degraded. When theresolution of the sensor 120 is reduced and the size of the sensingelement is maintained, the size of the sensor 120 and the focal length fof the lens 110 may be reduced. In this case, the size of the lens arraycamera 100 may be reduced but the resolution of the image may also bereduced.

The following example embodiments may provide technology related to alens array camera and a method of driving the lens array camera toaccurately restore a color image while satisfying a desired viewingangle, resolution, sensitivity, and size of the lens array camera 100.For example, by designing the lens 110 in a small size and maintainingthe size of the sensor 120 the focal length f of the lens 110 may bereduced and a thickness of the lens array camera 100 may also bereduced. Referring to FIG. 1, the lens array camera 100 may include thelens 110 and the sensor 120. The lens 110 and the sensor 120 included inthe lens array camera 100 of FIG. 1 will be described in detail withreference to FIGS. 2 and 3. A method of driving a sensing elementincluded in the sensor 120 will be described in detail with reference toFIG. 4.

The lens 110 may cover a predetermined area of the sensor 120corresponding to the size of the lens 110. In other words, light passingthrough each individual lens 110 may be incident on sensing elements ofthe sensor 120 included in the predetermined area. The light may includea plurality of light rays. Each of the sensing elements of the sensor120 may generate sensing information based on incident light rayspassing through the lens 110. For example, the sensing element maygenerate sensing information based on a light ray incident through thelens 110. The lens array camera 100 may acquire an image correspondingto viewpoints included in a field of view of the lens array camera 100based on the sensing information output by the sensor 120, restore theacquired image, and output a high-resolution image.

In this example, the number of the lenses 110 and the number of sensingelements included in the sensor 120 may be relatively prime. Thus, thelens 110 and the sensing element may be arranged in a fraction structurehaving a disparity by 1/N, N being the number of lenses. This featurewill be described in detail later.

A processor 130 may simultaneously drive sensing elements correspondingto similar original signal information among a plurality of sensingelements arranged in the fraction structure together with the lens 110.The sensing elements corresponding to the similar original signalinformation will be described in detail with reference to FIG. 4.

FIG. 2 is a diagram illustrating a relationship between a number ofsensing elements and a number of lenses according to an exampleembodiment.

A sensor 230 may receive light rays X1 through X7 corresponding toviewpoints 240. For example, as shown in FIG. 2, multiple light rays X1may correspond to a single viewpoint. The light rays X1 through X7 maybe detected by a sensor after passing through lenses 210 and 220. Thesensor 230 may include sensing elements S1 through S7 corresponding to afirst row among a plurality of rows. The following description will bemade based on the sensing elements S1 through S7.

The sensing elements S1 through S7 may sense the light rays X1 throughX7 having passed through a plurality of lenses and overlapping oneanother. For example, the sensing element S1 may generate sensinginformation by sensing the light rays X1 and X2 passing through a lens210. In this example, the generated sensing information may be used foroutputting a high-resolution image through an application of an imagerestoration algorithm.

The sensing information generated by the sensing elements S1 through S7may correspond to original signal information (i.e., incident light rayinformation from each of the viewpoints 240) as shown in Equation 1below.

S=T·X  [Equation 1]

In Equation 1, S denotes a matrix representing sensing informationsensed by each of the sensing elements. X denotes a matrix indicatingthe original signal information. T denotes a transformation matrix thatrepresents a relationship between sensing information detected by thesensing elements S1 through S7 and original signal informationcorresponding to incident light. The light rays X1 through X7, thelenses, and the sensing elements S1 through S7 of FIG. 2 may be modeledas shown in Equation 2 below.

$\begin{matrix}\begin{matrix}S & T & X \\{\begin{bmatrix}{S\; 1} \\{S\; 2} \\{S\; 3} \\{S\; 4} \\{S\; 5} \\{S\; 6} \\{S\; 7}\end{bmatrix} =} & \begin{bmatrix}1 & 1 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 1 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 1 & 1 & 0 \\1 & 0 & 0 & 0 & 0 & 0 & 1 \\0 & 1 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 1 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 1 & 1\end{bmatrix} & \begin{bmatrix}{X\; 1} \\{X\; 2} \\{X\; 3} \\{X\; 4} \\{\; {X\; 5}} \\{X\; 6} \\{X\; 7}\end{bmatrix}\end{matrix} & \lbrack {{Equation}\mspace{14mu} 2} \rbrack\end{matrix}$

For example, as set forth in Equation 2, the sensing element S1 maygenerate sensing information by sensing the light rays X1 and X2 passingthrough the lens 210. Likewise, the sensing element S2 may generatesensing information by sensing the light rays X3 and X4 passing throughthe lens 210 and the sensing element S3 may generate sensing informationby sensing the light rays X5 and X6 passing through the lens 210. Thesensing element S4 may generate sensing information by sensing the lightray X7 passing through the lens 210 and the light ray X1 passing throughthe lens 220. The sensing element S5 may generate sensing information bysensing the light rays X2 and X3 passing through the lens 220, thesensing element S6 may generate sensing information by sensing the lightrays X4 and X5 passing through the lens 220, and the sensing element S7may generate sensing information by sensing the light rays X6 and X7passing through the lens 220.

In this example, the light ray X1 may be sensed by both the sensingelement S4 and the sensing element S1, the light ray X3 may be sensed byboth the sensing element S5 and the sensing element S2, the light ray X5may be sensed by both the sensing element S6 and the sensing element S3,and the light ray X7 may be sensed by both the sensing element S7 andthe sensing element S4.

As such, to restore an original image corresponding to the viewpoint oflight rays X1, sensing information obtained by each of the sensingelement S1 and the sensing element S4 may be used. Likewise, sensinginformation obtained by each of the sensing element S2 and the sensingelement S5 may be used to restore an original image corresponding to theviewpoint of light rays X3, sensing information obtained by each of thesensing element S3 and the sensing element S6 may be used to restore anoriginal image corresponding to the viewpoint of light rays X5, andsensing information obtained by each of the sensing element S4 and thesensing element S7 may be used to restore an original imagecorresponding to the viewpoint of light rays X7.

The transformation matrix T between the sensing information generated bythe sensing elements S1 through S7 included in the sensor 230 andoriginal signal information corresponding to the light rays X1 throughX7 incident from each of the viewpoints 240 may be determined based onthe number of lenses and the number of sensing elements.

The fraction structure of the lens 110 and the sensing element will bedescribed. The number of lenses and the number of sensing elements maybe relatively prime. For example, the lens array camera may include N*Nlenses. Also, the lens array camera may include (N*M+1)(N*M+1) sensingelements. Thus, each lens may cover (M+1/N)(M+1/N) sensing elements. Inother words, each lens may further cover M sensing elements in theirentirety and a portion of one or more other sensing elementscorresponding to a disparity by 1/N. In other words, if the sensingelements are arranged in a grid shape, each lens may cover a fraction ofeach of the sensing elements positioned along one edge of the lens, andeach lens may cover the same fraction of each of the sensing elementspositioned along another edge of the lens perpendicular to the firstedge. The fraction covered by the lens may be an integer multiple of1/N.

For example, when a lens array camera includes 5*5 lenses and 26*26sensing elements, one lens may cover (5+⅕)(5+⅕) sensing elements. Thatis, the one lens may further cover sensing elements corresponding to adisparity by ⅕.

When the number of lenses and the number of sensing elements arerelatively prime, an inverse matrix of the transformation matrix T ofEquation 1 may exist. Since the inverse matrix of the transformationmatrix T exists, a matrix X indicating original signal information maybe calculated by multiplying the inverse matrix of the transformationmatrix T by a matrix S indicating sensing information detected by asensing element as shown in Equation 3 below.

X=T ⁻¹ ·S  [Equation 3]

Using the matrix X obtained from Equation 3, a high-resolution imagecorresponding to an original image may be output.

FIG. 3 is a diagram illustrating a relationship between a number ofsensing elements and a number of lenses according to an exampleembodiment.

A number of lenses and a number of sensing elements which are relativelyprime may satisfy Equation 4 below.

Number of sensing elements=Number of lenses(N)*naturalnumber(M)+1  [Equation 4]

As described with reference to FIG. 3, since a number of lenses N shownin the figure is 6 and a number of sensing elements in one row is 37, arelative prime relationship may be satisfied. In this example, 37/6sensing elements may be covered per lens.

For example, a first lens may cover the entireties of sensing elements 1through 6, and may additionally cover ⅙ of sensing element 7. A secondlens may cover the remaining ⅚ of sensing element 7, the entireties ofsensing elements 8 through 12, and 2/6 of sensing element 13. Likewise,a last lens may cover ⅙ of sensing element 31 and the entireties ofsensing elements 32 through 37. As such, each lens may further coversensing elements corresponding to a disparity by ⅙ (the number oflenses).

When Equation 4 is extended to two dimensions, the number of sensingelements in the entire grid may satisfy Equation 5 below.

Total number of sensing elements=(N*M+1)(N*M+1)  [Equation 5]

When the number of sensing elements in one row is 37, a number ofsensing elements in a second row may be 37 and, likewise, a number ofsensing elements in a 37th row may also be 37. Thus, each lens may cover(6+⅙)*(6+⅙) sensing elements and all lenses, for example, 6*6 lenses maycover 37*37 sensing elements.

According to an example embodiment, it is possible to output ahigh-resolution image close to an original image by restoring an imageacquired by a lens array camera including a lens and a sensing elementsatisfying Equation 5.

FIG. 4 is a diagram illustrating an arrangement state of a lens and asensing element according to an example embodiment.

Referring to FIG. 4, a lens array camera includes 5*5 lenses and 26*26sensing elements. In this example, a number of sensing elements in eachrow may be (5*5+1) and satisfy Equation 3, and a total number of sensingelements in the grid may be (5*5+1)(5*5+1) and satisfy Equation 4. Thus,one lens may cover (5+⅕)(5+⅕) sensing elements.

A first row through a 26^(th) row may be arranged in a row direction. 26sensing elements may be arranged in each of the first row through the26^(th) row. For example, 26 sensing elements may be arranged in thefirst row, 26 sensing elements may be arranged in a second row, and 26sensing elements may be arranged in the 26^(th) row.

Sensing elements corresponding to similar original signal informationmay be simultaneously driven. Here, the sensing elements correspondingto similar original signal information may be sensing elements sensing alight ray corresponding to the similar original signal information. Inthe example of FIG. 2, the sensing element S1 and the sensing element S4correspond to the same original light information because they bothsense the light rays X1. Also, in the example of FIG. 2, the sensingelement S2 and the sensing element S5 correspond to the same originalsignal information because they both sense the light rays X3.

As an example, in FIG. 4, (the 1^(st) row, a 6^(th) row, an 11^(th) row)may include sensing elements sensing light rays from the same viewpointand may thus correspond to similar original signal information. Thesensing elements of the 1^(st) row may sense a light ray passing througha first lens. The sensing elements of the 6^(th) row may overlap thefirst lens by ⅕ and overlap a second lens by ⅘. In other words, ⅕ ofeach sensing element in the 6^(th) row is covered by the first lens and⅘ of each sensing element in the 6^(th) row is covered by the secondlens. Thus, the sensing elements of the 6^(th) row may sense light rayspassing through both the first lens and the second lens. Likewise, thesensing elements of the 11^(th) row may overlap the second lens by ⅖ andoverlap a third lens by ⅗. In other words, ⅖ of each sensing element inthe 11^(th) row is covered by the second lens and ⅗ of each sensingelement in the 11^(th) row is covered by the third lens. Thus, thesensing elements of the 11^(th) row may sense light rays passing throughboth the second lens and the third lens. Although the sensing elementsof (the 6^(th) row, the 11^(th) row) overlap other lenses, the sensingelements may sense light rays corresponding to lowermost portions of thesecond lens and the third lens. Thus, the sensing elements of (the6^(th) row, the 11^(th) row) may sense light rays from the sameviewpoint and may thus correspond to similar original signal informationto that of the first row.

In this example, an overlap may occur when the number of lenses and thenumber of sensing elements satisfy Equation 4 and Equation 5. Forexample, the lens array camera may include 5*5 lenses and (5*5+1)(5*5+1)sensing elements. In this example, one lens may cover (5+⅕)(5+⅕) sensingelements. Through this, the overlap may occur due to a disparity by ⅕.

The sensing elements of a 16^(th) row may overlap the third lens by ⅗and overlap a fourth lens by ⅖. Thus, the sensing elements of the16^(th) row may sense light rays passing through both the third lens andthe fourth lens. Likewise, the sensing elements of a 21^(st) row mayoverlap the fourth lens by ⅘ and overlap a fifth lens by ⅕. Thus, thesensing elements of the 21^(st) row may sense light rays passing throughboth the fourth lens and the fifth lens. However, instead ofcorresponding to lowermost portions of the fourth and fifth lenses, thesensing elements of (the 16^(th) row, the 21^(st) row) may sense lightrays corresponding to uppermost portions of the third lens and thefourth lens. Thus, the sensing elements of (the 16^(th) row, the 21^(st)row) may not sense light rays from the same viewpoint and thus may notcorrespond to the original signal information similar to that of (the1^(st) row, the 6^(th) row, the 11^(th) row). Therefore, the sensingelements of (a 17^(th) row, a 22^(nd) row) subsequent to the sensingelements of (the 16^(th) row, the 21^(st) row) may sense light rays fromthe same viewpoint and may thus correspond to the original signalinformation similar to that of (the 1^(st) row, the 6^(th) row, the11^(th) row).

The lens array camera may drive the sensing elements included in (the1^(st) row, the 6^(th) row, the 11^(th) row, the 17^(th) row, the22^(nd) row) instead of (the 1^(st) row, the 6^(th) row, the 11^(th)row, the 16^(th) row, the 21^(st) row) to sense light rays correspondingto a lowermost portion of a lens.

As another example, in FIG. 4, (a 2^(nd) row, a 7^(th) row, a 12^(th)row) may include sensing elements sensing light rays from the sameviewpoint and may thus correspond to similar signal originalinformation. Similarly, rather than (the 17^(th) row, the 22^(nd) row),(a 18^(th) row, a 23^(rd) row) may include sensing elements sensinglight rays from the same viewpoint and may thus correspond to originalsignal information similar to that of (the 2^(nd) row, the 7th row, the12^(th) row).

Generalizing the description above, when a lens array camera includesN*N lenses and (N*M+1)(N*M+1) sensing elements, one lens may cover(M+1/N)(M+1/N) sensing elements. In this example, when p is equal to N/2rounded up to the nearest natural number, rows corresponding to r, M+r,2*M+r, (p−1)*M+r, p*M+r+1, . . . , (N−1)*M+r+1 may be simultaneouslydriven. In other words, the rows of sensing elements that may besimultaneously driven are given by:

$\begin{matrix}{\quad\{ \begin{matrix}{{{i \times M} + r},} & {0 \leq i < p} \\{{{i \times M} + r + 1},} & {p \leq i < N}\end{matrix} } & \lbrack {{Equation}\mspace{14mu} 6} \rbrack\end{matrix}$

The natural number r is incremented by 1 for each set of rows to besimultaneously driven. For example, r=1 for the first set of rows to besimultaneously driven.

The lens array camera may simultaneously drive the sensing elementsincluded in (the 1^(st) row, the 6^(th) row, the 11^(th) row, the17^(th) row, the 22^(nd) row), and then simultaneously drive the sensingelements included in (the 2^(nd) row, the 7^(th) row, the 12^(th) row,the 18^(th) row, the 23^(rd) row). The lens array camera maysimultaneously drive the sensing elements corresponding to similaroriginal signal information for each row instead of sensing on arow-by-row basis.

According to an example embodiment, a lens array camera including a lensand a sensing element satisfying Equation 5 may acquire an image asdescribed above. The image may be restored, so that a high-resolutionimage close to an original image is output.

FIG. 5 is a flowchart illustrating a lens array camera driving methodperformed by a lens array camera according to an example embodiment.

In operation 510, a lens array camera including N*N lenses and aplurality of sensing elements simultaneously drives sensing elementscorresponding to similar original signal information among a pluralityof sensing elements.

A number of the plurality of sensing elements and a number of the lensesmay be relatively prime. For example, the number of the plurality ofsensing elements may be a predetermined natural multiple of N+1, N beingthe number of lenses. Hereinafter, it is assumed that the number oflenses is N*N and p is equal to N/2 rounded up to the nearest naturalnumber.

Here, the sensing elements corresponding to the similar original signalinformation may include sensing elements at a same position relative tothe lens covering the sensing elements among a plurality of sensingelements in a case of a first lens through a p^(th) lens and sensingelements at the same position+1 in a row direction instead of thesensing elements at the same position in a case of a lens after andincluding a (p+1)^(th) lens.

In other words, the sensing elements may be divided into groups ofsensing elements. For each lens, every group may include one sensingelement that is covered by the lens. The sensing elements in a givengroup are each positioned at a same position or at a same position+1relative to the corresponding lens as set forth above to correspond to asame original signal viewpoint.

When p is equal to N/2 rounded up to the nearest natural number, eachgroup of sensing elements includes a plurality of sensing elements givenby Equation 6.

Also, after the sensing elements corresponding to the similar originalsignal information (i.e., all of the sensing elements in a given group)are simultaneously driven, the sensing elements located at theposition+1 in the row direction (i.e., all of the sensing elements inthe next group) may be repetitively and simultaneously driven.

For example, the number of sensing elements is 26*26, the number oflenses is 5*5, and a natural number rounded by 5/2, p is 3, sensingelements at a same position among a plurality of sensing elementscorresponding to each lens in a case of a first lens through a thirdlens and sensing elements at the same position+1 in a case of a fourthlens and a fifth lens may be simultaneously driven. Related descriptionis made with reference to FIG. 4.

As such, among the sensing elements corresponding to the first lensthrough the fifth lens, the sensing elements at the same position maynot be simultaneously driven and thus, a difference occurs therebetween.This is because the lens more covers the sensing elements by M+1/N dueto a disparity by 1/N occurring when the sensing element and the lensare arranged in a fraction structure described above and as shown, e.g.,in FIG. 4.

According to an example embodiment, an image acquired using a lens arraycamera including a lens and a sensing element satisfying the fractionstructure may be restored so as to be output as a high-resolution imagewith a high degree of similarity to an original image.

FIGS. 6 and 7 are diagrams illustrating a device in which a lens arraycamera is implemented according to an example embodiment.

An image acquired using a lens array camera may be applied to variousfields of outputting a high-resolution image. The lens array camera mayinclude a plurality of sensing elements spaced apart in a relativelysmall focal length through a plurality of lenses. The lens array cameramay have a small thickness.

Due to a use of the plurality of lenses, a size of the lens array cameramay be reduced. Thus, the lens array camera may be applied not only to auser terminal but also to a wearable device, such as a smart watch, asmart band, smart glasses, and the like, of which a size is relativelyimportant.

For example, as illustrated in FIG. 6, a lens array camera 610 may beimplemented in a user terminal 600 as a front or rear camera. A sensorof the lens array camera 610 may be implemented as a full-frame sensor.A lens of the lens array camera 610 may be implemented as a micro lens.

Referring to FIG. 7, a vehicle 700 may include lens array cameras atpoints, for example, points 720 and 730. A lens array camera may beadjustable in size and thus, may be installed in the vehicle withouthindering a design or stability.

For example, a lens array camera may be implemented as a front camera ora rear camera of the vehicle 700. In this example, the lens array cameramay use a curved lens array 710. That is, the curved lens array in whicha connecting portion between lenses is designed to be bent may be usedin the lens array camera.

However, the present examples are not to be taken as being limitedthereto. The lens array camera may be applicable to, for example, adigital single-lens reflex (DSLR) camera, a drone, a closed-circuittelevision (CCTV), a webcam camera, a panoramic camera, a movie orbroadcast video camera, and a virtual reality/augmented reality (VR/AR)camera. Furthermore, the lens array camera may be applicable to variousfields, for example, a flexible/stretchable camera, a compound-eyecamera, and a contact lens type camera.

The apparatuses, units, modules, devices, and other components describedherein are implemented by hardware components. Examples of hardwarecomponents that may be used to perform the operations described in thisapplication where appropriate include controllers, sensors, generators,drivers, memories, comparators, arithmetic logic units, adders,subtractors, multipliers, dividers, integrators, and any otherelectronic components configured to perform the operations described inthis application. In other examples, one or more of the hardwarecomponents that perform the operations described in this application areimplemented by computing hardware, for example, by one or moreprocessors or computers. A processor or computer may be implemented byone or more processing elements, such as an array of logic gates, acontroller and an arithmetic logic unit, a digital signal processor, amicrocomputer, a programmable logic controller, a field-programmablegate array, a programmable logic array, a microprocessor, or any otherdevice or combination of devices that is configured to respond to andexecute instructions in a defined manner to achieve a desired result. Inone example, a processor or computer includes, or is connected to, oneor more memories storing instructions or software that are executed bythe processor or computer. Hardware components implemented by aprocessor or computer may execute instructions or software, such as anoperating system (OS) and one or more software applications that run onthe OS, to perform the operations described in this application. Thehardware components may also access, manipulate, process, create, andstore data in response to execution of the instructions or software. Forsimplicity, the singular term “processor” or “computer” may be used inthe description of the examples described in this application, but inother examples multiple processors or computers may be used, or aprocessor or computer may include multiple processing elements, ormultiple types of processing elements, or both. For example, a singlehardware component or two or more hardware components may be implementedby a single processor, or two or more processors, or a processor and acontroller. One or more hardware components may be implemented by one ormore processors, or a processor and a controller, and one or more otherhardware components may be implemented by one or more other processors,or another processor and another controller. One or more processors, ora processor and a controller, may implement a single hardware component,or two or more hardware components. A hardware component may have anyone or more of different processing configurations, examples of whichinclude a single processor, independent processors, parallel processors,single-instruction single-data (SISD) multiprocessing,single-instruction multiple-data (SIMD) multiprocessing,multiple-instruction single-data (MISD) multiprocessing, andmultiple-instruction multiple-data (MIMD) multiprocessing.

The methods that perform the operations described in this applicationare performed by computing hardware, for example, by one or moreprocessors or computers, implemented as described above executinginstructions or software to perform the operations described in thisapplication that are performed by the methods. For example, a singleoperation or two or more operations may be performed by a singleprocessor, or two or more processors, or a processor and a controller.One or more operations may be performed by one or more processors, or aprocessor and a controller, and one or more other operations may beperformed by one or more other processors, or another processor andanother controller. One or more processors, or a processor and acontroller, may perform a single operation, or two or more operations.

Instructions or software to control a processor or computer to implementthe hardware components and perform the methods as described above arewritten as computer programs, code segments, instructions or anycombination thereof, for individually or collectively instructing orconfiguring the processor or computer to operate as a machine orspecial-purpose computer to perform the operations performed by thehardware components and the methods as described above. In one example,the instructions or software include machine code that is directlyexecuted by the processor or computer, such as machine code produced bya compiler. In another example, the instructions or software includehigher-level code that is executed by the processor or computer using aninterpreter. Programmers of ordinary skill in the art can readily writethe instructions or software based on the block diagrams and the flowcharts illustrated in the drawings and the corresponding descriptions inthe specification, which disclose algorithms for performing theoperations performed by the hardware components and the methods asdescribed above.

The instructions or software to control a processor or computer toimplement the hardware components and perform the methods as describedabove, and any associated data, data files, and data structures, arerecorded, stored, or fixed in or on one or more non-transitorycomputer-readable storage media. Examples of a non-transitorycomputer-readable storage medium include read-only memory (ROM),random-access programmable read only memory (PROM), electricallyerasable programmable read-only memory (EEPROM), random-access memory(RAM), dynamic random access memory (DRAM), static random access memory(SRAM), flash memory, non-volatile memory, CD-ROMs, CD-Rs, CD+Rs,CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs,BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, blue-ray or optical disk storage,hard disk drive (HDD), solid state drive (SSD), flash memory, a cardtype memory such as multimedia card micro or a card (for example, securedigital (SD) or extreme digital (XD)), magnetic tapes, floppy disks,magneto-optical data storage devices, optical data storage devices, harddisks, solid-state disks, and any other device that is configured tostore the instructions or software and any associated data, data files,and data structures in a non-transitory manner and providing theinstructions or software and any associated data, data files, and datastructures to a processor or computer so that the processor or computercan execute the instructions.

While this disclosure includes specific examples, it will be apparent toone of ordinary skill in the art that various changes in form anddetails may be made in these examples without departing from the spiritand scope of the claims and their equivalents. The examples describedherein are to be considered in a descriptive sense only, and not forpurposes of limitation. Descriptions of features or aspects in eachexample are to be considered as being applicable to similar features oraspects in other examples. Suitable results may be achieved if thedescribed techniques are performed in a different order, and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner, and/or replaced or supplemented by othercomponents or their equivalents. Therefore, the scope of the disclosureis defined not by the detailed description, but by the claims and theirequivalents, and all variations within the scope of the claims and theirequivalents are to be construed as being included in the disclosure.

What is claimed is:
 1. A method of driving a lens array camera, themethod comprising: simultaneously driving a first group of sensingelements from among a plurality of sensing elements, each sensingelement from among the first group of sensing elements corresponding toa same original signal viewpoint, wherein the plurality of sensingelements is included in a sensor corresponding to the lens array cameraincluding N rows of N lenses, N being a natural number.
 2. The method ofclaim 1, wherein a number of the plurality of sensing elements and anumber of the lenses are relatively prime.
 3. The method of claim 2,wherein the number of the plurality of sensing elements is one more thana predetermined natural multiple of N, N being the number of rows oflenses.
 4. The method of claim 1, wherein the plurality of sensingelements are arranged in (N*M+1) rows, M being a predetermined naturalnumber, wherein each row of the plurality of sensing elements includes(N*M+1) sensing elements, wherein when p is equal to N/2 rounded up tothe nearest natural number, the first group of sensing elements includesa plurality of rows of sensing elements given by$\quad\{ \begin{matrix}{{{i \times M} + r},} & {0 \leq i < p} \\{{{i \times M} + r + 1},} & {p \leq i < N}\end{matrix} $ wherein r is a natural number corresponding to thefirst group of sensing elements.
 5. The method of claim 4, wherein thefirst group of sensing elements is one from among a plurality of groupsof sensing elements, and wherein the natural number r is incremented by1 for each group of sensing elements from among the plurality of groupsof sensing elements to be simultaneously driven.
 6. The method of claim1, wherein each lens from among the N rows of N lenses covers a fractionof at least one sensing element from among the plurality of sensingelements that is less than an entirety of the at least one sensingelement.
 7. The method of claim 6, wherein the fraction of the at leastone covered sensing element is an integer multiple of 1/N.
 8. The methodof claim 1, further comprising: outputting an image corresponding tooriginal signal information received by the plurality of sensingelements by restoring sensing information obtained using thesimultaneously driven first group of sensing elements.
 9. Anon-transitory computer-readable storage medium storing instructionsthat, when executed by a processor, cause the processor to perform themethod of claim
 1. 10. A lens array camera comprising: a processor; anda memory including instructions to be read by a computer, wherein whenthe instructions are executed in the processor, the processor isconfigured to simultaneously drive a first group of sensing elementsfrom among a plurality of sensing elements, each sensing element fromamong the first group of sensing elements corresponding to a sameoriginal signal viewpoint, the plurality of sensing elementscorresponding to the lens array camera including N rows of N lenses, Nbeing a natural number.
 11. The lens array camera of claim 10, wherein anumber of the plurality of sensing elements and a number of the lensesare relatively prime.
 12. The lens array camera of claim 11, wherein thenumber of the plurality of sensing elements is one more than apredetermined natural multiple of N, N being the number of rows oflenses.
 13. The lens array camera of claim 10, wherein the plurality ofsensing elements are arranged in (N*M+1) rows, M being a predeterminednatural number, wherein each row of the plurality of sensing elementsincludes (N*M+1) sensing elements, wherein when p is equal to N/2rounded up to the nearest natural number, the first group of sensingelements comprises a plurality of rows of sensing elements given by$\quad\{ \begin{matrix}{{{i \times M} + r},} & {0 \leq i < p} \\{{{i \times M} + r + 1},} & {p \leq i < N}\end{matrix} $ wherein r is a natural number corresponding to thefirst group of sensing elements.
 14. The lens array camera of claim 13,wherein the first group of sensing elements is one from among aplurality of groups of sensing elements, and wherein the natural numberr is incremented by 1 for each group of sensing elements from among theplurality of groups of sensing elements to be simultaneously driven. 15.The lens array camera of claim 10, wherein each lens from among the Nrows of N lenses covers a fraction of at least one sensing element fromamong the plurality of sensing elements that is less than an entirety ofthe at least one sensing element.
 16. The lens array camera of claim 15,wherein the fraction of the at least one covered sensing element is aninteger multiple of 1/N.
 17. The lens array camera of claim 10, whereinthe processor is further configured to output an image corresponding tooriginal signal information received by the plurality of sensingelements by restoring sensing information obtained using thesimultaneously driven first group of sensing elements.